Show the warnings the compiler already had - #467
Merged
Conversation
A pipeline could print
✓ p.yaml is valid
while carrying a warning that a reference could not be checked, and then fail
at run time on exactly that reference (#465). The compiler had collected the
finding all along; nothing displayed it. It went to the log stream, which
validate does not write to and a script capturing stdout never sees.
A successful validate now prints its warnings, with their codes, and says how
many in the summary line. Informational findings stay out: 'tool is available'
and 'execution order computed' are not things anyone needs told, and reporting
that would turn the channel into noise.
--json emits the whole result as one document, findings included, with field
names that are an interface rather than English to be parsed: code, severity,
category, step, parameter_path, referenced_step, referenced_field, message,
suggestions. Data-flow findings now carry the referenced step and field
structurally instead of only in prose.
Along the way: cli.py defined 'def list()' at module scope, rebinding the
builtin for the whole file. A click Command is callable, so 'list(...)' did
not build a list -- it invoked the command, printed the configured providers
and exited before its own output. --json emitted nothing and looked like a
registration problem. The function is now list_keys with the CLI name passed
to the decorator, and a test refuses any module-level name in cli.py that
shadows a builtin.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test_keys_list_is_still_reachable ran 'orchestrator keys list', which reports the user's configured providers and exits non-zero when there are none. It passed on my machine because I have keys and failed in CI because CI has none -- by design; the default workflow is hermetic and carries no secrets. The test was asserting the wrong thing anyway. What matters is that renaming the function to list_keys did not rename the CLI command, and that is a question about registration: it is now asked of the click group directly, and an e2e companion uses --help, which resolves the command without needing any credential to exist. Verified by running the suite with the provider variables unset and HOME pointed at an empty directory, so ~/.orchestrator/.env is absent too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The second half of #465.
The defect
A pipeline could print
while carrying a warning saying a reference could not be checked — and then fail at run time on exactly that reference. The compiler had collected the finding all along; nothing displayed it. It went to the log stream, which
validatedoes not write to and a script capturing stdout never sees.As the review put it: logging is not an adequate user-facing warning channel.
The human channel
Informational findings stay out. "Tool is available" and "execution order computed" are not things anyone needs told, and printing them would turn the channel into noise — at which point it stops being read, which is how this started.
Warnings do not fail validation. Making them errors would reject pipelines that run correctly, which is the false-positive class removed in #448/#450/#461.
The structured channel
--jsonemits the whole result as one document — and only that document, since anything appended makes it unparseable, a mistake caught in the catalogue report earlier this week.Field names are the interface, so nothing has to parse English to learn what is wrong:
{ "code": "data_flow_undefined_output", "severity": "warning", "category": "data_flow", "step": "use", "parameter_path": "content", "referenced_step": "make", "referenced_field": "path", "message": "...", "suggestions": [] }referenced_stepandreferenced_fieldare new: data-flow findings carried them only inside the prose message. JSON is emitted for invalid pipelines too, so a consumer does not switch parsers based on the outcome.A latent hazard found on the way
--jsoninitially printed the configured provider list and exited, emitting no JSON at all. It looked like a click registration problem;--jsonwas registered correctly.cli.pydefineddef list()at module scope, rebinding the builtin for the whole file. A click Command is callable, solist(issue.suggestions or [])did not build a list — it invoked thekeys listcommand, printed the providers, and exited before its own output.Fixed at the root: the function is
list_keyswith the CLI name passed to the decorator, so the command name is unchanged. A test now refuses any module-level name incli.pythat shadows a builtin, because anything named after a builtin here is one call away from repeating this.Verification
keys liststill works under its CLI name, with a test.🤖 Generated with Claude Code